{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "---\n",
    "title: DiffPose\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Intraoperative 2D/3D registration via differentiable X-ray rendering\n",
    "\n",
    "[![CI](https://github.com/eigenvivek/DiffPose/actions/workflows/test.yaml/badge.svg)](https://github.com/eigenvivek/DiffPose/actions/workflows/test.yaml)\n",
    "[![Paper shield](https://img.shields.io/badge/arXiv-2312.06358-red.svg)](https://arxiv.org/abs/2312.06358)\n",
    "[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)\n",
    "[![Docs](https://github.com/eigenvivek/DiffPose/actions/workflows/deploy.yaml/badge.svg)](https://vivekg.dev/DiffPose)\n",
    "[![Code style: black](https://img.shields.io/badge/Code%20style-black-black.svg)](https://github.com/psf/black)\n",
    "\n",
    "![](experiments/test_time_optimization.gif)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install\n",
    "\n",
    "To install `DiffPose` and the requirements in [`environment.yml`](https://github.com/eigenvivek/DiffPose/blob/main/environment.yml), run:\n",
    "\n",
    "```zsh\n",
    "pip install diffpose\n",
    "```\n",
    "\n",
    "The differentiable X-ray renderer that powers the backend of `DiffPose` is available at [`DiffDRR`](https://github.com/eigenvivek/DiffDRR)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Datasets\n",
    "\n",
    "We evaluate `DiffPose` networks on the following open-source datasets:\n",
    "\n",
    "| **Dataset**                                                                | **Anatomy**        | **\\# of Subjects** | **\\# of 2D Images** | **CTs** | **X-rays** | Fiducials |\n",
    "|----------------------------------------------------------------------------|--------------------|:------------------:|:-------------------:|:-------:|:----------:|:---------:|\n",
    "| [`DeepFluoro`](https://github.com/rg2/DeepFluoroLabeling-IPCAI2020)        | Pelvis             |          6         |         366         |    ✅    |      ✅     |     ❌     |\n",
    "| [`Ljubljana`](https://lit.fe.uni-lj.si/en/research/resources/3D-2D-GS-CA/) | Cerebrovasculature |         10         |          20         |    ✅    |      ✅     |     ✅     |\n",
    "<!-- | [`2D-3D-GS`](https://lit.fe.uni-lj.si/en/research/resources/2D-3D-GS/)     | Lumbar Spine       |          1         |          18         |    ✅    |      ✅     |     ✅     |\n",
    "| [`VerSe`](https://github.com/anjany/verse)                                 | Spine              |         355        |         N/A         |    ✅    |      ❌     |     ❌     | -->\n",
    "\n",
    "- `DeepFluoro` ([**Grupp et al., 2020**](https://link.springer.com/article/10.1007/s11548-020-02162-7)) provides paired X-ray fluoroscopy images and CT volume of the pelvis. The data were collected from six cadaveric subjects at John Hopkins University. Ground truth camera poses were estimated with an offline registration process. A visualization of one X-ray / CT pair in the `DeepFluoro` dataset is available [here](https://vivekg.dev/DiffPose/experiments/render.html).\n",
    "\n",
    "```zsh\n",
    "mkdir -p data/\n",
    "wget --no-check-certificate -O data/ipcai_2020_full_res_data.zip \"http://archive.data.jhu.edu/api/access/datafile/:persistentId/?persistentId=doi:10.7281/T1/IFSXNV/EAN9GH\"\n",
    "unzip -o data/ipcai_2020_full_res_data.zip -d data\n",
    "rm data/ipcai_2020_full_res_data.zip\n",
    "```\n",
    "\n",
    "- `Ljubljana` ([**Mitrovic et al., 2013**](https://ieeexplore.ieee.org/abstract/document/6507588)) provides paired 2D/3D digital subtraction angiography (DSA) images. The data were collected from 10 patients undergoing endovascular image-guided interventions at the University of Ljubljana. Ground truth camera poses were estimated by registering surface fiducial markers.\n",
    "\n",
    "```zsh\n",
    "mkdir -p data/\n",
    "wget --no-check-certificate -O data/ljubljana.zip \"https://drive.google.com/uc?export=download&confirm=yes&id=1x585pGLI8QGk21qZ2oGwwQ9LMJ09Tqrx\"\n",
    "unzip -o data/ljubljana.zip -d data\n",
    "rm data/ljubljana.zip\n",
    "```\n",
    "\n",
    "<!-- - `2D-3D-GS` ([**Tomaževič et al., 2004**](https://pubmed.ncbi.nlm.nih.gov/16192053/)) ...\n",
    "\n",
    "- `VerSe` ([**Sekuboyina et al., 2020**](https://pubs.rsna.org/doi/10.1148/ryai.2020190074)) ... -->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Experiments\n",
    "\n",
    "To run the experiments in `DiffPose`, run the following scripts (ensure you've downloaded the data first):\n",
    "\n",
    "```zsh\n",
    "# DeepFluoro dataset\n",
    "cd experiments/deepfluoro\n",
    "srun python train.py     # Pretrain pose regression CNN on synthetic X-rays\n",
    "srun python register.py  # Run test-time optimization with the best network per subject\n",
    "```\n",
    "\n",
    "```zsh\n",
    "# Ljubljana dataset\n",
    "cd experiments/ljubljana\n",
    "srun python train.py\n",
    "srun python register.py\n",
    "```\n",
    "\n",
    "The training and test-time optimization scripts use SLURM to run on all subjects in parallel:\n",
    "\n",
    "- `experiments/deepfluoro/train.py` is configured to run across six A6000 GPUs\n",
    "- `experiments/deepfluoro/register.py` is configured to run across six 2080 Ti GPUs\n",
    "- `experiments/ljubljana/train.py` is configured to run across twenty 2080 Ti GPUs\n",
    "- `experiments/ljubljana/register.py` is configured to run on twenty 2080 Ti GPUs\n",
    "\n",
    "The GPU configurations can be changed at the end of each script using [`submitit`](https://github.com/facebookincubator/submitit)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Development\n",
    "\n",
    "`DiffPose` package, docs, and CI are all built using [`nbdev`](https://nbdev.fast.ai/).\n",
    "To get set up with`nbdev`, install the following\n",
    "\n",
    "```zsh\n",
    "conda install jupyterlab nbdev -c fastai -c conda-forge \n",
    "nbdev_install_quarto      # To build docs\n",
    "nbdev_install_hooks       # Make notebooks git-friendly\n",
    "pip install -e  \".[dev]\"  # Install the development verison of DiffPose\n",
    "```\n",
    "\n",
    "Running `nbdev_help` will give you the full list of options. The most important ones are\n",
    "\n",
    "```zsh\n",
    "nbdev_preview  # Render docs locally and inspect in browser\n",
    "nbdev_clean    # NECESSARY BEFORE PUSHING\n",
    "nbdev_test     # tests notebooks\n",
    "nbdev_export   # builds package and builds docs\n",
    "nbdev_readme   # Render the readme\n",
    "```\n",
    "\n",
    "For more details, follow this [in-depth tutorial](https://nbdev.fast.ai/tutorials/tutorial.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Citing `DiffPose`\n",
    "\n",
    "If you find `DiffPose` or [`DiffDRR`](https://github.com/eigenvivek/DiffDRR) useful in your work, please cite the appropriate papers:\n",
    "\n",
    "```\n",
    "@misc{gopalakrishnan2022diffpose,\n",
    "    title={Intraoperative 2D/3D Image Registration via Differentiable X-ray Rendering}, \n",
    "    author={Vivek Gopalakrishnan and Neel Dey and Polina Golland},\n",
    "    year={2023},\n",
    "    eprint={2312.06358},\n",
    "    archivePrefix={arXiv},\n",
    "    primaryClass={cs.CV}\n",
    "}\n",
    "\n",
    "@inproceedings{gopalakrishnan2022diffdrr,\n",
    "    author={Gopalakrishnan, Vivek and Golland, Polina},\n",
    "    title={Fast Auto-Differentiable Digitally Reconstructed Radiographs for Solving Inverse Problems in Intraoperative Imaging},\n",
    "    year={2022},\n",
    "    booktitle={Clinical Image-based Procedures: 11th International Workshop, CLIP 2022, Held in Conjunction with MICCAI 2022, Singapore, Proceedings},\n",
    "    series={Lecture Notes in Computer Science},\n",
    "    publisher={Springer},\n",
    "    doi={https://doi.org/10.1007/978-3-031-23179-7_1},\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python3",
   "language": "python",
   "name": "python3"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
